Systems and methods for implementing hailing request

ABSTRACT

Systems and methods for assisting a user to hail a vehicle using a gesture or application. In one aspect, when a vehicle detects a user hails it with a gesture, it calculates a route and moves to the user autonomously. In another aspect, after a vehicle receives a message from a remote facility, it starts searching for a user nearby and moves to the user autonomously when the user hails it via a gesture or application.

FEDERALLY SPONSORED RESEARCH

Not applicable

SEQUENCE LISTING OR PROGRAM

Not applicable

BACKGROUND Field of Invention

This invention relates to hailing a vehicle, more particularly tohailing an autonomous vehicle.

Description of Prior Art

Autonomous vehicle (also known as driverless or self-driving vehicle) isa vehicle capable of sensing and navigating around the vehicle'ssurroundings and travelling autonomously to a destination without userinput. It represents a great advance in the transportation industry.

When autonomous vehicles become widely available, many users may rely onself-driving taxis or self-driving cargo vehicles to go to places orship goods. Usually a user places an order online and then waits for aservice to dispatch a vehicle. Sometimes, there are vehicles parkedbeside the curb, sitting at a parking lot, or running on a road. When auser sees a vehicle with a vacant sign, the user may want to hail itdirectly on site, instead of contacting a remote facility and goingthrough an online ordering or reservation process.

Therefore, there exists a need for a user to hail an autonomous vehicledirectly when the vehicle is in sight.

As used herein, word “vehicle” may mean any form of motorizedtransportation or a device capable of moving around autonomously.Examples of vehicle may include automobile, drone, flying car, aircraft,ship, robot, and the like. For convenience of description, “ServiceCenter” as used herein may mean a center or remote facility as abusiness entity or a server which is operated at Service Center. “Checkin” as used herein may mean a user logs in a system at a vehicle usinginfo obtained from a reservation or using other suitable info. In onescenario, after a check-in process, the identity of a user may beconfirmed. In another scenario, after a check-in process, the identityof a user may remain unknown. “Hailing (hail) a vehicle” and “Hail acar” as used herein may mean a physical act, a verbal act, or an onlineprocess which is performed by a user for ordering a vehicle for rentingpurpose or reserving a vehicle for present or future use.

Objects and Advantages

Accordingly, several main objects and advantages of the presentinvention are:

-   -   a). to provide improved systems and methods for hailing a        vehicle;    -   b). to provide such systems and methods for a user to hail a        vehicle directly when the vehicle is in sight;    -   c). to provide such systems and methods which enable a user to        hail a vehicle with gestures;    -   d). to provide such systems and methods which enable a user to        find vehicles nearby; and    -   e). to provide such systems and methods which enable a vehicle        to recognize a user's gesture and drive to the user        autonomously.

Further objects and advantages will become apparent from a considerationof the drawings and ensuing description.

SUMMARY

In accordance with the present invention, vehicle hailing systems andmethods are provided. When a user sees an autonomous vehicle, the usermay hail it directly using a gesture. On the other hand, when anautonomous vehicle detects that a user is gesturing at it, the vehiclemay calculate a route and navigate to the user. In addition, a user mayfind vehicles nearby via an app and select a vehicle with gestures.Using gestures to hail a vehicle is natural, intuitive, and convenientwhen the vehicle is visible in a short distance.

DRAWING FIGURES

FIG. 1-A is an exemplary diagram describing an embodiment involving avehicle, a client system, and a server in accordance with the presentinvention.

FIG. 1-B is an exemplary diagram which depicts a client system inaccordance with the present invention.

FIG. 1-C is an exemplary diagram which depicts a user and a vehicle inaccordance with the present invention.

FIGS. 2-A and 2-B are exemplary diagrams which illustrate a hailingscenario in accordance with the present invention.

FIG. 3 is an exemplary flow diagram depicting a hailing process inaccordance with the present invention.

FIGS. 4 and 5 are exemplary diagrams which illustrate interface settingsfor hailing a vehicle in accordance with the present invention.

FIGS. 6 and 7 are exemplary diagrams which illustrate interface settingsfor finding vehicles in pointing mode in accordance with the presentinvention.

FIGS. 8 and 9 are exemplary flow diagrams depicting hailing processes inaccordance with the present invention.

FIG. 10 is an exemplary flow diagram depicting vehicle activities inaccordance with the present invention.

FIGS. 11 and 12 are exemplary diagrams showing hailing processes inaccordance with the present invention.

FIGS. 13 and 14 are exemplary flow diagrams describing hailing processesin accordance with the present invention.

REFERENCE NUMERALS IN DRAWINGS

10 Processor 12 Computer Readable Medium 14 Communication Network 16Client System 18 Vehicle 20 Processing Module 22 Database 24 Server 26Control System 28 User 30 Vehicle 32 Vehicle 34 Vehicle 36 User 38Smartphone 40 Button 42 Button 44 Button 46 Screen 48 Window 50 Button52 Window 54 Button 56 Route 58 Button 60 Window 62 Window 64 Button 66Button 100-172 are exemplary steps.

DETAILED DESCRIPTION

The following exemplary embodiments are provided for complete disclosureof the present invention and to fully inform the scope of the presentinvention to those skilled in the art, and the present invention is notlimited to the schematic embodiments disclosed, but can be implementedin various types.

FIG. 1-A is an exemplary block diagram of one embodiment according tothe present invention. A vehicle 18 and server 24 are connected via awireless communication network 14. So are a client system 16 and server24. Vehicle 18 is controlled by a control system 26. Server 24 isinstalled and operated at Service Center. Service Center represents aride-hailing business. It administers vehicles including vehicle 18 andprocesses hailing requests from users. The word “server” as used hereinmeans a system or systems which may have similar functions andcapacities as one or more servers. Server 24 may exemplarily be dividedinto two blocks, represented by a processing module 20 and database 22.Processing module 20 may include processing and communication functions.Database 22 may store vehicle service records and information, map dataand geographic info of certain areas, user account information, usertransaction records, etc. The database may include a cluster of memorychips and/or storage modules. In the figure, servers 24 may represent adevice that collects, processes, stores, and maintains information anddocuments, sends instructions to vehicles, transmits messages to users,executes tasks requested by users, etc.

FIG. 1-B describes client system 16 exemplarily. Client system 16 maycover a range of electronic devices and gadgets, e.g., a smartphone, atablet computer, a smart watch, a virtual reality (VR) device, anaugmented reality (AR) device, and the like. Client system 16 mayinclude a processor 10 and computer readable medium 12. Processor 10 maymean one or more processor chips or systems. Medium 12 may be the mainpart of a storage system and may include a memory hierarchy built by oneor more memory chips or storage components like RAM, ROM, FLASH, orother suitable storage modules. Processor 10 may run programs or sets ofexecutable instructions stored in medium 12 for performing variousfunctions and tasks like surfing on the Internet, placing purchaseorders, hailing a vehicle, sending and receiving emails and shortmessages, playing video or music, etc. Client system 16 may also includeinput, output, and communication components, which may be individualmodules or integrated with processor 10.

In addition, client system 16 may have a display (not shown in thefigure for brevity reason) and a graphical user interface (GUI). Thedisplay may have a liquid crystal display (LCD) screen or light emittingdiode (LED) screen and may be arranged sensitive to touches, i.e.,sensitive to haptic and/or tactile contact with a user. A user may usethe interface to interact with client system 16, Service Center orserver 24, an e-commerce website, other online users, and so on. Via theinterface, for instance, a user may hail a vehicle, place online orders,play a game, communicate with other devices, obtain all sorts ofinformation, and so on.

Furthermore, client system 16 may have a voice recognition device toreceive a user's verbal command or voice input. The system may also havefacial recognition capability to recognize a user. In addition, thesystem may have a gesture detection mechanism to receive a user'sgesture instructions. For VR and AR devices and some wearable devices, avirtual screen or a screen with a very small size may be arranged. Avirtual screen may be part of a displaying system that doesn't have aphysical screen. Since it is impractical or inconvenient to touch avirtual screen or very small screen, a user may use voice and gesturesto interact with a system and issue commands.

FIG. 1-C shows that a user 28, who is making a hailing gesture, isdetected by vehicle 18 schematically. Vehicle 18 may be a device whichmoves around autonomously. Examples of vehicle 18 include automobile,robot, drone, flying car, ship, or the like. For simplicity reasons, anautonomous automobile is used as vehicle 18 to explain principles. Thevehicle may have various sensors (not shown in the figure) to ascertainthe external environment and internal situation. The sensors may includecameras, a radar system, a light detection and ranging (LIDAR) system, aglobal position system (GPS), a speed sensor, an accelerometer, anelectronic compass, etc. Control system 26 of vehicle 18, like clientsystem 16, may have a computer processor, a storage device, a displaywith a touch sensitive screen, a voice recognition system, a facialrecognition system, and a gesture detection system. It is noted thatvoice, facial, and gesture recognition technologies are all maturenowadays. The control system may use the sensors to monitor theenvironment, detect a user, interact with a user, and receiveinstructions from a user. Moreover, the control system may calculate aroute after receiving a user's hailing request and navigate vehicle 18to the user along the route. In FIG. 1-C. control system 26 finds andascertains user 28 via sensors like cameras, radar, and LIDAR. Itanalyzes the user's gestures and determines that the user is waving atvehicle 18.

When a user needs a vehicle and sees one on a street, it is natural andconvenient to hail the vehicle with a gesture directly. FIGS. 2-A and2-B use exemplary diagrams to illustrate a hailing process. In FIG. 2-A,there are three vehicles 30, 32, and 34. Vehicles 32 and 34 have nosigns and are running toward opposite directions, while vehicle 30 isparked beside the curb with a vacant sign which indicates it is vacantand available for hailing. A user 36 finds vehicle 30 and starts wavingat it. In FIG. 2-B, a control system of vehicle 30 detects that user 36is waving. It analyzes the user's gestures via certain algorithm andconcludes that the user is hailing it. Then the control system sends ahailing message to Service Center and asks for approval. Once getting anapproval message, the control system calculates a route 56 and drivesvehicle 30 to user 36 along the route. Alternatively, the control systemmay also navigate vehicle 30 to approach user 36 without gettingapproval from Service Centre. It is noted that the method applies to avehicle running on a road as well.

FIG. 3 shows an exemplary flow diagram which illustrates a hailingprocess. A vehicle is in standby mode and displays a vacant sign. Thevacant sign may have a word like “Vacant” and is arranged to show thevehicle is available for hailing. A control system of the vehiclemonitors its surroundings using cameras and other sensors at step 100.The system finds that a user is waving at the vehicle at step 102. Todetermine whether a user waves at a vehicle, there are two conditions.First, the user waves his or her hand with the palm facing the vehicle.Second, the user is facing the vehicle. If it is detected that a userwaves and the two conditions are satisfied at the same time, it isconsidered that the user waves at the vehicle. If it is detected that auser waves but the two conditions are not satisfied simultaneously, thewaving act is ignored. After determining the user is waving at thevehicle, the control system contacts Service Center and sends a hailingmessage at step 104. Service Center reviews the hailing message andchecks whether the vehicle is assigned to a job already. The center alsochecks whether there is another vehicle which sends a similar hailingmessage regarding the same user in a given time period. If the vehicleis not reserved for any assignment and the vehicle is the only one whichsends a hailing message within the time period, Service Center sends anapproval message to the control system.

After getting approval from Service Center at step 106, the controlsystem continues monitoring the user at step 108. The user may walk tothe vehicle or stay put at the same place. If it is detected that theuser walks toward the vehicle, the vehicle doesn't need to move andhence it waits for the user at the same spot. The control system mayreplace the vacant sign by another sign with a word like “Welcome”, whenthe user walks to the vehicle. The user may start a check-in processafter getting in the vehicle at step 110. If the user doesn't movetoward the vehicle, the control system may drive the vehicle to the userafter a predetermined time period elapses. The control system maymeasure the position of the user and calculate a route to go there atstep 112. Next at step 114, the control system starts a motor or engineand navigates the vehicle to approach the user, while replacing thevacant sign by a sign with a word like “Coming”. After the vehicle stopsin front of the user, the user gets in the vehicle and proceeds withcheck-in procedures at step 110. At step 116, the control systemcalculates a route to a destination place and begins a journey. Afterthe user gets in, the vehicle may display a sign with a word like“Occupied” so that other users know it is not available for hailing.

Sometimes, e.g., during the rush hour, more than one user may wave at avehicle at the same time. Thus certain rules may be made for cases whenmultiple users waves simultaneously. For instance, if a user and avehicle are on the same side of a road while the other user is on theother side, the user on the same side prevails. When multiple users areon the same side of a road, the user who is closer or the closest to thevehicle prevails. For such a purpose, a vehicle may be arranged tomonitor and analyze multiple users at the same time.

When more than one vehicle responds to a waving user, the user may use afinger or a hand to point at a vehicle to select it. When a vehicledetects that a user points or waves at another vehicle or object, itwill not proceed to pick up the user.

It is noted that systems and methods for hailing a vehicle not onlyapply to automobiles, but also apply to robots, drones, flying cars, andships, as they may have equivalent sensors to ascertain a user and maymaneuver to the user autonomously.

A user may gesture at a vehicle when the vehicle is in sight andmonitoring the user. But a hailing gesture doesn't work if a vehicle isin sleeping mode with sensors turned off. When a vehicle is in sleepingmode, a user may use an app to contact Service Center and let the centersend a message to activate the vehicle. FIG. 4 shows an exemplarydiagram of an app interface which a user may use to hail a car or find acar. Assume that Car App is installed at a smartphone 38. The app may beprovided by Service Center for hailing or reserving a vehicle. As in thefigure, the app is launched and content in the app interface is shown ona touch-sensitive screen 46. There are three interactive elements orbuttons 40, 42, and 44 representing three options presented to a user.The buttons have exemplary labels like “Hail Car”, “Find Car around Me”,and “Schedule a Pickup” respectively. The top portion of the interfaceis configured for a simple and quick hailing process which requires oneaction only. The first line shows a title, like a sentence “1-Tap toHail a Car”. Below the title, button 40 is arranged for starting aone-tap-to-hail-a-car process.

The app is designed such that a user may just tap button 40 to completea vehicle hailing process without submitting a pickup location, pickuptime, destination location, or other information. Assume that a user hasan account at Service Center and enables a location option at phone 38.When the location option is enabled, phone 38 may turn on GPS or useanother positioning mechanism to get its location info. After Car App islaunched, an interface like the one in FIG. 4 appears. Since the phone'slocation is measured constantly, the app may obtain data of the currentlocation anytime. Once a user taps button 40, it prompts the app to senda message to Service Center. The message may contain at least threeitems: The user's account number, data of the current location, and arequest for a vehicle. Service Center may assume that the user needs avehicle right now and a pickup place may be the user's current locationor a nearby place if the current location is not suitable for pickup.Next Service Center may select and dispatch a vehicle to pick up theuser and send a confirmation message to the user in the meantime. Themessage may show up in the app interface with content like when avehicle will come and where a pickup place will be. If the user closesthe app after tapping button 40, Service Center may send the user anemail or a short message containing similar information.

As a user only needs to tap button 40 after the app is launched, hailinga vehicle may become simple, quick, and convenient. Such aone-tap-to-hail-a-car process especially fits autonomous vehicles. Forinstance, before a taxi with a driver is dispatched, the driver's workarea and work schedule have to be reviewed. Some drivers may not beavailable for a long trip or a trip to a specific area. On the contrary,autonomous vehicles don't have such limiting factors. They may betreated relatively equal. They may be sent to a user without knowingdestination info or without worrying about where the user will go.

When a user wants to schedule a pickup at a later time or at anotherlocation, the user may tap button 44. After the button is tapped, awindow may appear where related information may be submitted.

When a user needs a vehicle, the user may tap button 40 or 44 to hailone. Next, Service Center may dispatch a vehicle to pick up the user.Sometimes when there are available vehicles nearby, a user may want totake a look at the vehicles, compare them, and then choose one to hailit. Button 42 is arranged for checking vehicles within a given shortdistance. When the app senses button 42 is tapped, it sends a message toService Center. The message contains the user's location info and arequest for finding vehicles. The user's account number may or may notbe included, depending on the setting or user selection. Next, ServiceCenter receives the message and retrieves vehicle info from database.The center may conduct a search and select vehicles which are parkedaround the user within a given distance. The given distance may bedetermined by Service Center and arranged around, for instance, onehundred to two hundred meters. Service Center then sends info about theselected vehicles to the user. The info contains locations of selectedvehicles and a map covering areas around the user.

After getting the info from Service Center, the app presents it in theinterface, as depicted schematically in FIG. 5. A message “5 cars aroundyou” summarizes the search result. The map is presented in a window 48.There are five vehicle icons representing five vehicles on the map. Auser may take a look at the map and then search around to find thevehicles in the surroundings. The five vehicles may be notified byService Center already and each may show a vacant sign. A button 58 witha label “Hello” is configured beneath the map area. Once button 58 isactivated, the app sends a “Hello” message to Service Center. The centerthen sends messages to vehicles which are shown on the map. Afterreceiving the messages, the vehicles may respond by showing a “Hello”signal or sign. The “Hello” signal or sign may be arranged moreconspicuous than the vacant sign. It shows that a vehicle is availableand ready for hailing. Examples of the “Hello” signal or sign mayinclude flashes of certain light and a word like “Hello”, “Available”,or “Ready”, etc. The “Hello” signal or sign may last for a short periodof time, such as around one to three minutes. A user may tap button 58to have a better survey on nearby vehicles. When the user finds asuitable vehicle, the user may hail it with a gesture or via the app.

Alternatively, a small check box may be configured beside each vehicleicon on the map of window 48. A user may check one or more boxes andthen tap “Hello” button 58. Then only the checked vehicles, i.e., thevehicles whose boxes are checked, are notified by Service Center. Thesevehicles may respond by showing the “Hello” signal or sign. Thus, a usermay use the check boxes and the “Hello” button to recognize each vehicleindividually. For instance, a user may check a box, tap the “Hello”button, and then look around to see which vehicle shows the “Hello”signal or sign. The check box helps a user find and interact with avehicle conveniently.

The vehicle icons which represent vehicles on the map are arrangedinteractive. A user may tap a vehicle icon to find info about a vehicle.For instance, a window may show up after a vehicle icon is tapped. Thewindow may provide vehicle info like make, model, year, color, and soon. The window may also present an interactive button “Hail this car”.When a user taps “Hail this car” button, the app sends a message toService Center. The message may state that the user has selected thevehicle and wants to hail it. Next, Service Center may approve therequest and assign the vehicle to the user. The center may send amessage to the user to confirm the hailing result and send anothermessage to notify the vehicle.

Since button 40 remains valid in the app interface, a user may tap itanytime to do a quick hailing process. It is noted that tapping button40 means a user asks Service Center to choose a vehicle. Service Centermay select a vehicle around the user or from another place.

Although positions of available vehicles are displayed on a map, such asthe map in window 48, some users may still find it hard to know wherethe vehicles are in the real world. The problem is that a traditionalmap shows locations in all directions and it is not intuitive to match aplace on a map to a place in front of a user. In order to overcomedifficulties to figure out a vehicle's real location, a button 50 isarranged. The button, having a label “Pointing Mode”, is used to presentan elongated map in the interface.

After sensing button 50 is tapped, the app removes window 48 andreplaces it with a window 52, as illustrated schematically in FIG. 6.Window 52 shows an elongated map. A message “2 cars ahead of you” tellsa user that there are two vehicles in a forward direction. Anothermessage “Point Mode” is used to remind a user that the map is inpointing mode. “Pointing mode” as used herein means a map has anelongated shape and is elongated along the phone's pointing direction.When a screen of a smartphone or smart watch is in a horizontal plane,the pointing direction is what its front end faces at. When the screenis in a vertical plane, the pointing direction is what the screen's backfaces at. As the map in window 52 is narrow, it may be arranged todisplay available vehicles only in front of a user or only in a forwarddirection and still within a short given distance. When there are twovehicle icons on the map, it means there are two vehicles in thepointing direction of phone 38. Hence confusion about which direction avehicle is located in may be overcome, since the direction is what thephone points at. It is noted that the pointing mode needs orientationdata of a device. Thus orientation capability should be enabled at adevice to support the pointing mode.

Similar to button 58, a “Hello” button 54 is configured on screen 46 ofFIG. 6. A user may tap button 54 to send a “Hello” message to ServiceCenter. The center then sends messages to the two vehicles and lets themshow the “Hello” signal or sign.

In addition, the user may also points phone 38 at different directionsto check vehicle availability elsewhere. FIG. 7 depicts a scenario whenthere is no available vehicle along a pointing direction.Correspondingly, there is no vehicle icon on the elongated map in window52. In such a case, button 54 may be configured to appear blank with adecreased brightness level.

Returning to FIG. 4. When a user wants to find vehicles around him orher, the user may tap button 42. After that, the app sends a searchrequest to Service Center. The request also contains location data ofthe user. The center then searches available vehicles nearby. Next, thecenter sends to the user location info of selected vehicles and a mapwhich covers a given short distance from the user. In the meantime, thecenter sends wake-up messages to the selected vehicles which appear onthe map.

Once a vehicle receives a wake-up message from Service Center, it startsa hailing process. FIG. 8 shows an exemplary flow diagram for such anevent. At step 118, a vehicle receives a wake-up message from ServiceCenter. Next, the vehicle begins searching for a user who may want tohail a vehicle at step 120. The vehicle may already have a vacant signto show it's available. If not, it displays the sign to notify peoplearound it. As Service Center has location data of the user, it passesthe info to the vehicle so that the vehicle knows where the user mightbe. Resultantly, the vehicle may ignore users from other areas. Thusinterference, misunderstanding, and misrepresentation may be reduced.

Sometimes the vehicle may not find the user immediately, as the user maybe blocked by an object, e.g., a building or another vehicle. If asearch has no result at step 122, step 120 is taken again and the searchcontinues. When the vehicle finds the user at step 122, it ascertainswhether the user waves at it at step 124. If the user doesn't wave atthe vehicle within a given time period, it may mean the user may not bethe person who wants to hail a vehicle. Then the vehicle returns to step120 and starts another search. If it is detected that the user waves atthe vehicle at step 124, the vehicle sends a hailing message to ServiceCenter and asks for approval at step 126. Once receiving an approvalmessage, the vehicle calculates a route and drives to the user at step130. If it is not approved, the process ends at step 128. Asaforementioned, Service Center may use the approval process to verifythat the vehicle is not reserved for another task. The approval processis also useful when multiple vehicles each determine a user is waving atit. As a user may gesture from a distance, it may be difficult to tellwhich vehicle the user aims at. Then Service Center may receive hailingrequests from several vehicles within a given time frame, say one tofive seconds. The center may select a vehicle based on certain rules.For instance, the center may pick a vehicle which is the closest to theuser and assign it to take the task. Alternatively, the center mayselect a vehicle which is the first to submit a hailing request.

In addition, the vehicle may ascertain whether the user walks toward itat step 124. If the user waves or raises hand and then walks to thevehicle, the vehicle may stay there waiting for the user to come.

FIG. 9 shows another exemplary flow diagram of a hailing process. Avehicle receives a wake-up message from Service Center at step 132. Asaforementioned, after a user launches an app at a user device andrequests a vehicle search at Service Center, the center sends wake-upmessages to selected vehicles which are within a given short distancefrom the user. One of the selected vehicles starts a process to find theuser at step 134. The vehicle may already have a vacant sign to showit's available. If not, it displays the sign to notify the public.Meanwhile, Service Center also sends relevant vehicle info to the user.An app interface like that of FIG. 5 may show up at the user device.Vehicle icons representing available vehicles within the short distanceare displayed.

At step 136, the vehicle monitors messages from Service Center. One ofthe messages may contain hailing instructions. For instance, a user maytap a vehicle icon in the interface, like the icons in window 48 or 52of FIG. 5 or 6. After the app detects that a vehicle icon is tapped, itpresents a new window. The window may show info of the vehicle and a“Hail this car” button. Then the user taps the “Hail this car” button toreserve the vehicle. In response to activation of the button, the appcontacts Service Center and passes the info. Service Center then assignsthe vehicle to the user and sends hailing instructions to the vehicle.After the vehicle receives the hailing instructions, it drives to theuser autonomously at step 142 and picks up the user.

If the vehicle doesn't receive any hailing instructions at step 136, itsearches for the user or detects whether any hailing gestures are madewhen the user is found at step 138. If the user is not found or hailinggestures are not observed for a given period of time, the vehiclereturns to step 134 to start another round. If hailing gestures aredetected at step 138, the vehicle sends a hailing message to ServiceCenter. At step 140, if the vehicle gets a disapproval message from thecenter, the hailing process ends at step 141. If the vehicle receivesapproval from the center, it drives to the user autonomously at step 142and picks up the user.

FIG. 10 shows an exemplary flow diagram which depicts vehicle activitiesaccording to the present invention. When a vehicle is free of anyassignment and sits beside the curb of a road or at a parking lot, itmay have two modes. At step 144, a vehicle is in standby mode with avacant sign displayed. The vehicle's engine or motor is turned off,while the sensors which monitor the external environment are still on.The vehicle is arranged to monitor nearby users, ascertain them, anddetect their gestures at step 150. If a user waves at the vehicle, acontrol system of the vehicle may analyze it, determine the meaning ofit, and drives to the user autonomously with or without approval fromService Center.

Alternatively, a vehicle may also rest in sleeping mode as shown at step146. When a vehicle is in sleeping mode, not only the engine or motor isoff, but also most sensors are off. The vehicle may display a sign withwords like “Out of service”. The vehicle doesn't monitor itssurroundings or detect a user's gestures. The sleeping mode may beinterrupted by a signal from Service Center at step 148. After thevehicle receives a wake-up signal or wake-up message from ServiceCenter, it turns on the sensors and displays a vacant sign. The vehiclethen starts monitoring users in the surroundings at step 150. Hence, auser may gesture to hail a vehicle in standby mode, but may not do it ifthe vehicle is in sleeping mode. A vehicle in sleeping mode may beactivated by Service Center, which may happen after the center receivesa search request from a user.

FIG. 11 shows an exemplary app interface which reflects a scenario aftera user hails a vehicle according to the present invention. The vehiclemay be hailed via a gesture or tapping a button as aforementioned. Atitle “Vehicle Coming” is displayed on the top portion of the screen,which assures the user that the vehicle is coming to pick up him or her.A window 60 may show the make, model, year, color of the vehicle andother info. A map in a window 62 may show the real-time position of thevehicle and the user. Location info of the vehicle may come from ServiceCenter, which gets it from the vehicle. The user may tap a “Yes” button64 to verify the hailing result. If the user doesn't tap button 64, itwould not matter. If the user finds a wrong vehicle is selected, theuser may tap a “No, not this one” button 66 to cancel the selection.After button 66 is tapped, the app may notify Service Center and presentother available vehicles on the map. Then the user may tap an icon ofanother vehicle to select it. The user may also gestures at anothervehicle to hail it, as discussed in the above. The newly selectedvehicle may appear moving on the map as it approaches the user.

If the user walks to the vehicle after hailing it, the vehicle maydetect the action and report it to Service Center. Service Center thensends a message to the app. The app in turn presents another interface,which is depicted exemplarily in FIG. 12. The title of the interfacesays “Going to Vehicle”, confirming that the user is going to thevehicle. Overall, FIG. 12 resembles FIG. 11. Information about thevehicle is presented in window 60. The map in window 62 may show thatthe user moves toward the vehicle in real time, while the vehicle staysput. Similarly, buttons 64 and 66 are arranged for the user to verifythe hailing act or cancel it.

After detecting a user's hailing gestures, a vehicle may drive to theuser directly or wait for approval from Service Center beforeapproaching the user. Both methods have merits. The former is easy toimplement, while the latter may improve user experience as it preventsconfusion and awkwardness caused by multiple vehicles going to the sameuser. FIG. 13 shows an exemplary flow diagram which describes a hailingprocess supervised by Service Center. At step 152, Service Centerreceives a hailing message from a vehicle. The message is sent after thevehicle detects that a user makes a hailing gesture at it. The centerchecks contents of the message. At step 154, the center obtains from themessage location data of the vehicle, location data of the user, andinfo about the gestures which are considered as a hailing request.Location data of the user is acquired by the vehicle via measurementsusing sensors. The center may also obtain from the message images of theuser which are taken by the vehicle.

At step 156, Service Center reviews the hailing case. Based on locationinfo of the user, or the location info plus images of the user, thecenter checks how many vehicles send a hailing message about the sameuser. When a user is at a distance while some vehicles are relativelyclose to each other, the vehicles may receive the same gesture signalsfrom the user. Thus more than one vehicle may receive the same hailingrequest. When Service Center receives a hailing message from only onevehicle within a given time period, the vehicle is selected. WhenService Center receives hailing messages from multiple vehicles withinthe given time period, the center may select one based on predeterminedrules. For instance, the center may calculate a distance between avehicle and the user based on the location info. A vehicle which is theclosest to the user may be selected. Alternatively, the center may alsoselect a vehicle which is the first to send a hailing message. Inaddition, the center may check whether a vehicle is reserved already andremove the vehicle if it is assigned to another task. At step 158, avehicle is chosen and assigned to the user after the reviewing process.The center then sends an approval message to the vehicle. The methodapplies to vehicles sitting at a place or running on a road. When thevehicles are parked, the given time period may be one to five seconds.When the vehicles are running, the given time period may be one to twoseconds or shorter.

FIG. 14 shows another exemplary flow diagram about a hailing processsupervised by Service Center. At step 160, Service Center receives avehicle search request from a user. The request is sent after the userlaunches an app at a user device and submits a search request, liketapping button 42 of FIG. 4. As aforementioned, location data of theuser is also sent to the center with the search request. At step 162,the center finds suitable vehicles around the user within a given shortdistance and retrieves info of the selected vehicles from database. Atstep 164, the center sends the vehicle info to the user. Once the userdevice obtains the vehicle info, it presents a map in the app interface,where locations of the selected vehicles are displayed. At step 168,Service Center receives a hailing request for a vehicle from the user.As discussed, the user may tap a vehicle icon on the map and proceed tohail the vehicle. The center may review the request, approve it, andsend messages to the user and the chosen vehicle respectively at step172. The message to the user confirms the hailing act. The message tothe vehicle contains location data of the user and instructions for thevehicle to find and engage the user.

After receiving a search request from the user, Service Center alsosends messages or wake-up messages to the selected vehicles at step 166.When a vehicle is in standby mode, the message serves as a notification.When a vehicle is in sleeping mode, the message works as a trigger or anactivation command besides a notification. As discussed, the selectedvehicles may search for the user and display vacant signs to show theyare available. Once a vehicle detects that the user gestures at it, itsends a hailing message to Service Center. The center may receive thehailing message from one or more vehicles at step 170. The center thenreviews the hailing case, chooses a vehicle based on certain selectionrules when multiple vehicles are involved, and assigns the vehicle tothe user. As aforementioned, the center may select a vehicle based onvehicle-user distance or message-sending time. At step 172, ServiceCenter sends a confirmation message to the user and instructions to theassigned vehicle to conclude the hailing application process.

CONCLUSION, RAMIFICATIONS, AND SCOPE

Thus it can be seen that systems and methods are introduced for hailinga vehicle.

The systems and methods have the following main features and advantages:

-   -   (1) A user may hail a vehicle by gestures when seeing it;    -   (2) A user may hail a nearby vehicle via an app or via an app        plus gestures;    -   (3) A user may search vehicles within a short distance using an        app;    -   (4) A vehicle may monitor a user and detect hailing gestures;    -   (5) A vehicle may navigate to a user autonomously after the user        hails it; and    -   (6) A vehicle may stay at a place waiting for a user after the        user hails it and then walks toward it.

Although the description above contains many specificities, these shouldnot be construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodiments.Numerous modifications will be obvious to those skilled in the art.

RAMIFICATIONS

A vehicle may detect and recognize a gesture through certain algorithmand convert the gesture into instructions. Besides waving a hand,raising a hand, and using a finger to point at a vehicle, other handsigns, such as a “Come here” hand signal, may also be used as hailinggestures. While gesturing, a user has to face the vehicle; otherwise,the gestures are ignored. When a user is close to a vehicle, the usermay also nod to contact, select, or hail a vehicle. Nodding may beespecially useful for contacting or hailing a robot, as a user may meeta robot at a close distance. In addition, gazing may be combined withnodding to reduce miscommunication. For instance, a vehicle may startgaze sensing procedures when it detects a user nods at it. Gaze sensingmay be turned off when nodding is not detected or certain conditions arenot satisfied, as users prefer less surveillance. Optionally it may beconfigured that only when nodding and gazing acts happen at the sametime, the nodding act is recognized. Service Center may decide to usenodding alone or nodding plus gazing as a command. As aforementioned,vehicles as used herein include automobiles, robots, drones, flyingcars, and ships. Thus hailing gestures apply to all of them.

A facial recognition system of a vehicle may be used to identify a userand match the user with an account in records during a check-in process.Thus a user may submit a picture to Service Center and then check in avehicle without using code, fingerprint, credit card, or cash. A facialrecognition system may also be used to estimate the age of anunregistered user. For instance, a child may not be allowed to hail avehicle. When a vehicle detects that a person gestures at it, itoperates a facial recognition system to ascertain whether the person isa child. If the person is a child, the vehicle stops the hailingactivity. It may protect children and prevent pranks. Rules about how todetermine a person is a child may be defined by Service Center. Inaddition, images of a user may also be used to estimate the age. Forinstance, a user's height, contour, and cloths may be analyzed for ageestimation.

In above discussions, a user taps a button to enter an input andinteract with a user device. Alternatively, the user may also speak tothe device. The device may use a voice recognition system to take andinterpret voice commands. For instance, a user may say “Yes” or “No” tophone 38 of FIGS. 11 and 12, which may be arranged to have the sameeffect as tapping button 64 or 66. A user may also say one or a fewwords which have the meaning of yes or no, like “Good”, “Okay”, “Comehere”, “I'm waiting”, “No, wrong car”, etc.

When a control system of a vehicle has a voice recognition mechanism, auser may also speak to the vehicle when his or her voice is audible toit. For instance, a user may say certain key words like “Taxi”, “You”,or “I need a vehicle” to select or hail a vehicle. As a vehicle may havegaze detection capability, a gazing requirement may apply. For instance,a user may have to look at a vehicle when issuing a verbal command. If avehicle detects that a user doesn't look at it while saying something,the user's verbal input may be ignored. Alternatively, a gazingrequirement may be replaced by a facing requirement, i.e., a user mayalso face a vehicle while speaking to it. Service Center may decidewhether to implement the gazing or face requirement. A user may saywords or a phrase like “Taxi”, “I Need a vehicle”, or “Come here” tohail a vehicle with or without making hailing gestures. A vehicle maykeep monitoring the surroundings and especially people around it. Afterreceiving a user's voice request, the vehicle sends a message to ServiceCenter for approval. Once getting an approval message from the center,the vehicle calculates a route and approaches the user autonomously.

A user may also use an AR device to hail a vehicle. For instance, a usermay open an app and then say “Show cars around me”. After the AR devicesends a request to Service Center and receives vehicle info from thecenter, it displays a map to show available vehicles nearby. If the usersays “Pointing mode”, the app replaces a regular map by an elongatedmap. For AR devices, the pointing direction is the direction which auser faces at. Thus when a user looks ahead, an AR device may show anelongated map on a virtual display. The map shows an area which is infront of the user.

Sometimes a user may change mind after hailing a vehicle and the vehicleis on the way to approach him or her. Regardless that the vehicle hasarrived or not, the user may cancel the hailing event. For instance, theuser may tap button 66 of FIG. 11 to cancel it as discussed. A user mayalso wave at a direction away from a coming vehicle. A vehicle maymonitor a hailing user continuously. Once the vehicle detects that theuser waves at another object for a given period of time, it maydetermine that the user cancels the reservation. If the vehicle detectsthat the user waves at another object briefly and then waves at itagain, it may consider the user still wants to keep the reservation. Thevehicle then continues moving toward the user. The same strategy appliesto a user walking to a vehicle. For instance, a user may stop walking toa target vehicle and wave at a direction away from the vehicle for acertain time period. After the vehicle detects the waving act, it maytreat it as a signal to cancel the reservation.

Lastly, a vehicle may be arranged to handle a hailing process with lessor no instructions from Service Center depending on prearrangements. Forinstance, after a vehicle detects that a user is waving at it, thevehicle may calculate a route and set forth a short journey to drive tothe user. During the hailing steps, Service Center is not involved. Themethod may be favored by small ride-hailing companies. In other cases, avehicle may maintain contact with Service Center and communicate to thecenter constantly. However, the vehicle may not need approval from thecenter during a hailing process. Service Center may work as an infosource and play a messenger role between the vehicle and a user.

Therefore the scope of the invention should be determined by theappended claims and their legal equivalents, rather than by the examplesgiven.

The invention claimed is:
 1. A method for a vehicle or device for ahailing process, comprising: 1) presenting a sign, wherein the signshows that the vehicle or device is vacant or ready for hailing; 2)ascertaining a user using one or more sensors, analyzing an act orgesture of the user, and determining whether the user waves or gesturesand whether the user faces the vehicle or device; 3) transmitting ahailing message to a service facility after determining that the userwaves or gestures and the user faces the vehicle or device, waiting foran approval message from the service facility after ascertaining theuser and transmitting the hailing message; 4) receiving from the servicefacility the approval message for the vehicle or device to pick up theuser, wherein the vehicle or device receives the approval message afterthe vehicle or device transmits the hailing message to the servicefacility; and 5) in response to receiving the approval message from theservice facility, calculating a route and navigating the vehicle ordevice to the user along the route after it is detected that the userdoes not move toward the vehicle or device within a predetermined timeperiod or waiting for the user after it is detected that the user movestoward the vehicle or device.
 2. The method according to claim 1,further including searching for the user using information received fromthe service facility before ascertaining the user.
 3. The methodaccording to claim 1, further including presenting a response message tothe user after it is detected that the user does not move toward thevehicle or device within the predetermined time period or the user movestoward the vehicle or device.
 4. The method according to claim 1,further including stopping navigating the vehicle or device to the userafter detecting that the user waves or gestures at a direction nottoward the vehicle or device.
 5. The method according to claim 1 whereinthe gesture includes a waving, pointing, or nodding act.
 6. The methodaccording to claim 1 wherein the sign is presented after receivinginformation that the user wants to hail a vehicle or device from theservice facility.
 7. The method according to claim 1, further includingturning on the one or more sensors for ascertaining the user afterreceiving a signal from the service facility.
 8. A method for a vehicleor device to respond to a hailing request, comprising: 1) displaying asign, wherein the sign shows that the vehicle or device is vacant orready for hailing; 2) searching for a user who faces the vehicle ordevice and waves or gestures using one or more sensors after receivinginformation about the hailing request from a service facility; 3)transmitting a hailing message to the service facility after finding theuser who faces the vehicle or device and waves or gestures, waiting foran approval message from the service facility after searching for theuser and transmitting the hailing message; 4) receiving from the servicefacility the approval message for the vehicle or device to pick up theuser, wherein the vehicle or device receives the approval message afterthe vehicle or device transmits the hailing message to the servicefacility; and 5) calculating a route and navigating the vehicle ordevice to the user along the route after receiving the approval messagefrom the service facility.
 9. The method according to claim 8, furtherincluding searching for the user around a location provided in theinformation about the hailing request received from the servicefacility.
 10. The method according to claim 8, further including waitingfor the user when it is detected that the user moves toward the vehicleor device after receiving the approval message from the servicefacility.
 11. The method according to claim 8, further includingstopping navigating the vehicle or device to the user after detectingthat the user waves or gestures at a direction not toward the vehicle ordevice.
 12. The method according to claim 8 wherein a gesture which theuser makes includes a waving, pointing, or nodding act.
 13. The methodaccording to claim 8, further including presenting a response message tothe user in response to an action of the user.
 14. The method accordingto claim 8, further including turning on the one or more sensors forsearching for the user after receiving the information about the hailingrequest.
 15. A vehicle or device capable of moving around, comprising: aplurality of sensors; and a control system including a processor and amemory or memory unit, the control system operable to: 1) present asign, wherein the sign shows that the vehicle or device is vacant orready for hailing; 2) ascertain a user using at least one of theplurality of sensors, analyze an act or gesture of the user, anddetermine whether the user waves or gestures and whether the user facesthe vehicle or device; 3) transmit a hailing message to a servicefacility after determine that the user waves or gestures and the userfaces the vehicle or device, wait for an approval message from theservice facility after ascertaining the user and transmitting thehailing message; 4) receive from the service facility the approvalmessage for the vehicle or device to pick up the user, wherein thevehicle or device receives the approval message after the vehicle ordevice transmits the hailing message to the service facility; and 5) inresponse to receiving the approval message from the service facility,calculate a route and navigate the vehicle or device to the user alongthe route after it is detected that the user does not move toward thevehicle or device within a predetermined time period or wait for theuser after it is detected that the user moves toward the vehicle ordevice.
 16. The vehicle or device according to claim 15 wherein thecontrol system is arranged to search for the user after receivinginformation from the service facility.
 17. The vehicle or deviceaccording to claim 15 wherein a response message is presented to theuser in response to an action of the user.
 18. The vehicle or deviceaccording to claim 15 wherein the control system stops navigating thevehicle or device to the user after detecting that the user waves orgestures at a direction not toward the vehicle or device.
 19. Thevehicle or device according to claim 15 wherein the gesture includes awaving, pointing, or nodding act.
 20. The vehicle or device according toclaim 15 wherein the control system is arranged to turn on the at leastone of the plurality of sensors for ascertaining the user afterreceiving a signal from the service facility.